Update cerebras.ts #9024
Update cerebras.ts #9024
Conversation
Updated maxTokens, contextWindow and description.
dosubot
bot
added
size:XS
hannesrudolph
added
the
Issue/PR - Triage
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| "zai-glm-4.6": { | ||
| maxTokens: 16_384, | ||
| contextWindow: 128000, | ||
| maxTokens: 40960, |
There was a problem hiding this comment.
Hey @sebastiand-cerebras, I'm up for whatever you all think is best but personally I think reserving that many tokens for output is a little much for a 131k context window.
There was a problem hiding this comment.
Hi Matt,
Thank you so much for your message! We decided to go for this value across our external messaging but I see that it's rather a theoretical value. Would you want me to reduce it in the case of roocode? Wouldn't be an issue!
Btw, I am a big fan :)
There was a problem hiding this comment.
Thank you! Yeah personally I've found 16k to be enough for coding, and then you reserve ~115k for task history before condensing etc.
There was a problem hiding this comment.
I updated the maxTokens. What do you think?
3 participants
Description
Updated maxTokens, contextWindow and description.
Test Procedure
n/a
Pre-Submission Checklist
Documentation Updates
Additional Notes
Thank you so much!
Important
Update
zai-glm-4.6model incerebras.tswith increasedmaxTokens,contextWindow, and reviseddescription.cerebras.ts, updatedzai-glm-4.6model:maxTokensincreased from 16,384 to 40,960.contextWindowincreased from 128,000 to 131,072.descriptionupdated to "Highly intelligent general purpose model with up to 1,000 tokens/s".This description was created by
for 6260be2. You can customize this summary. It will automatically update as commits are pushed.